Buffering of Data Transfers for Direct Access Block Devices

ABSTRACT

Described embodiments provide a method of transferring, by a media controller, data associated with a host data transfer between a host device and a storage media. A buffer layer module of the media controller segments the host data transfer into one or more data transfer segments. Each data transfer segment corresponds to at least a portion of the data. The buffer layer module allocates a number of physical buffers to a virtual circular buffer for buffering the one or more data transfer segments. The buffer layer module transfers, by the virtual circular buffer, each of the data transfer segments between the host device and the storage media through the allocated physical buffers.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of the filing date of U.S.provisional application Nos. 61/245,112 filed Sep. 23, 2009, and61/245,973 filed Sep. 25, 2009, the teachings of which are incorporatedherein in their entireties by reference.

The subject matter of this application is related to U.S. patentapplication Ser. No. 12/436,227 filed May 6, 2009, Ser. No. 12/475,710filed Jun. 1, 2009, Ser. No. 12/475,716 filed Jun. 1, 2009, Ser. No.12/477,996 filed Jun. 4, 2009, Ser. No. 12/478,013 filed Jun. 4, 2009,Ser. No. 12/508,879 filed Jul. 24, 2009, Ser. No. 12/508,915 filed Jul.24, 2009, Ser. No. 12/643,471 filed Dec. 21, 2009, Ser. No. 12/649,490filed Dec. 30, 2009, Ser. No. 12/722,828 filed Mar. 12, 2010, Ser. No.12/730,627 filed Mar. 24, 2010, ______ filed ______, 2010, (300.181,300.182) and ______ filed ______, 2010 (300.242) the teachings of all ofwhich are incorporated herein in their entireties by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to memory storage devices, and, inparticular, to buffering of data transfers for direct access blockdevices, such as solid state disks (SSDs).

2. Description of the Related Art

Flash memory is a type of non-volatile memory that is electricallyerasable and re-programmable. Flash memory is primarily used in memorycards and USB flash drives for general storage and transfer of databetween computers and other digital products. Flash memory is a specifictype of electrically erasable programmable read-only memory (EEPROM)that is programmed and erased in large blocks. One commonly employedtype of flash memory technology is NAND flash memory. NAND flash memoryforms the core of the flash memory available today, especially forremovable universal serial bus (USB) storage devices known as USB flashdrives, as well as most memory cards. NAND flash memory exhibits fasterase and write times, requires small chip area per cell, and has highendurance. However, the I/O interface of NAND flash memory does notprovide full address and data bus capability and, thus, generally doesnot allow random access to memory locations.

There are three basic operations for NAND devices: read, write anderase. The read and write operations are performed on a page by pagebasis. Page sizes are generally 2^(N) bytes, where N is an integer, withtypical page sizes of, for example, 2,048 bytes (2 kb), 4,096 bytes (4kb), 8,192 bytes (8 kb) or more per page. Pages are typically arrangedin blocks, and an erase operation is performed on a block by blockbasis. Typical block sizes are, for example, 64 or 128 pages per block.Pages must be written sequentially, usually from a low address to a highaddress. Lower addresses cannot be rewritten until the block is erased.

A hard disk is addressed linearly by logical block address (LBA). A harddisk write operation provides new data to be written to a given LBA. Olddata is over-written by new data at the same physical LBA. NAND flashmemories are accessed analogously to block devices, such as hard disks.NAND devices address memory linearly by page number. However, each pagemight generally be written only once since a NAND device requires that ablock of data be erased before new data is written to the block. Thus,for a NAND device to write new data to a given LBA, the new data iswritten to an erased page that is a different physical page than thepage previously used for that LBA. Therefore, NAND devices requiredevice driver software, or a separate controller chip with firmware, tomaintain a record of mappings of each LBA to the current page numberwhere its data is stored. This record mapping is typically managed by aflash translation layer (FTL) in software that might generate alogical-to-physical translation table. The flash translation layercorresponds to the media layer of software and/or firmware controllingan HDD.

Associated with each page is a spare area (typically 100-500 bytes)generally used for storage of error correction code (ECC) informationand for storage of metadata used for memory management. The ECC isgenerally needed for detecting and correcting errors in the user datastored in the page, and the metadata is used for mapping logicaladdresses to and from physical addresses. As such, the additional bytesof memory are “hidden” from the user and are not available for storinguser data. The first block (block 0) of a flash die is generallyprovided from the manufacturer error-free, and is commonly used bydesigners to include program code and associated metadata for blockmanagement.

For consumer applications, HDDs generally have data sectors that aresized in powers of two (e.g. 512 (2⁹) bytes per sector). Flash memoriesstructured with page sizes that are a multiple of the HDD sector sizemight efficiently work with the HDD system by storing multiple entiresectors in a page (e.g. a 4096 byte page can store eight 512 bytesectors). However, enterprise-based HDD systems generally do not usesectors sized by powers of two, but use larger sectors, generally either520 or 528 bytes per sector instead of 512 bytes. Thus, typical flashmemories perform inefficiently for enterprise applications since thereare unused bytes in each page.

Typically, for high capacity solid state disks (SSDs), several designtradeoffs might be considered when implementing a method to maintain alogical-to-physical translation table. These tradeoffs typicallyinclude: efficient random access memory (RAM) usage; efficient flashusage; fast address lookup for both read operations and writeoperations; fast write performance; and fast reconstruction of thetranslation table on device startup.

Several techniques are known in the art for maintaining thelogical-to-physical translation table. One such approach is known asdirect page mapping, an example of which is described in the paper byAndrew Birrell & Michael Isard, et al., A DESIGN FOR HIGH-PERFORMANCEFLASH DISKS , ACM SIGOPS Operating Systems Review, Vol. 41, Issue 2, pp.88-93, (April 2007), which is incorporated herein by reference in itsentirety (hereinafter “Birrell”). Direct page mapping maintains a lookuptable in RAM having an entry for each flash page, and a summary page formetadata at the end of each block, from which the logical-to-physicaltranslation table may be reconstructed at startup. For example, a directpage mapped translation table might contain, for every LBA, a logicalsector number corresponding to a physical block number and a physicalpage number. Thus, direct page mapping comprises a single-levellogical-to-physical translation. The summary page for each block mightcontain the LBA and valid bits for each page in the block so that thetranslation table can be reconstructed at startup. Thus, the direct pagemapping scheme requires a large amount of RAM (on the order of 1-2 MBper GB of user storage) to store the translation table, which can becomeburdensome for higher capacity SSDs.

Another approach is known as block mapping. Block mapping generallyclassifies blocks as either data blocks (D-blocks) or update blocks(U-blocks). The total size of the D-blocks is the effective storagespace for user data while U-blocks are invisible to users. Generally,when a write command cannot be accommodated in the D-block correspondingto the LBA, a U-block is allocated to receive the new data and the olddata in the D-block is invalidated. Subsequent writes to that D-blockwill be received by the allocated U-block. When the U-block becomesfull, another U-block might be allocated, or the U-block might be mergedwith the original D-block. Thus, block mapping maintains a lookup tablein RAM that maps a logical block to a physical block. Block mappinglacks a page-level map, instead relying on the typical case that data isstored in sequential order within the block. For example, a block mappedtranslation table might contain a logical sector number corresponding toa logical block number and a logical page number. The logical blocknumber can be translated into a physical block number and the logicalpage number might correspond to a physical offset within the physicalblock. Thus, block mapping comprises a two-level logical-to-physicaltranslation. The size of the translation table is proportional to thenumber of blocks in the flash memory, thus requiring less RAM than apage mapped translation table.

However, because block mapping does not have a page-level map, the flashmedia may be inefficiently utilized when the data access workload isnon-sequential. For non-sequential data access workloads, block mappingmight require data to be copied and re-written numerous times tomaintain the correct mapping. An example of block mapping is describedin the paper by Jeong-Uk Kang & Heeseung Jo, et al., A SUPERBLOCK-BASEDFLASH TRANSLATION LAYER FOR NAND FLASH MEMORY, Proceedings of the 6thACM & IEEE International Conference On Embedded Software, pp. 161-170,(Oct. 22-25, 2006), which is incorporated herein by reference in itsentirety (hereinafter “Kang”).

A third approach for maintaining the logical-to-physical translationtable is known as a superblock mapping scheme. Superblock mapping groupstogether a set number of adjacent logical blocks into a Superblock.Superblock mapping maintains a page global directory (PGD) in RAM foreach Superblock. Page middle directories (PMDs) and page tables (PTs)are maintained in the spare areas of the flash pages. Each LBA can bedivided into a logical block number and a logical page number, with thelogical block number comprising a superblock number and a PGD indexoffset. The logical page number comprises a PMD index offset and a PTindex offset. Each entry of the PGD points to a corresponding PMD. Eachentry of the PMD points to a corresponding PT. The PT contains thephysical block number and the physical page number of the data. Totranslate a logical address to a physical address in Superblock mapping,a module must access RAM to read the PGD, access flash to read the PMD,access flash to read the PT, and access flash to access the requesteddata address. Super-block mapping, thus, comprises a four-levellogical-to-physical translation and provides page-mapping.

The PMD's and PT's are stored in the spare areas of the flash pages toprovide page-mapping without using an excessive amount of RAM. However,because the spare area is used to store page-level mapping information,less memory is available for error correction codes (ECC). Further, thelimited amount of memory available in the spare area precludes storingcomplicating mapping information. Finally, reconstruction of thetranslation table at startup can be time-intensive. An example of asuperblock mapping scheme is described in Kang.

As described previously, for write operations, NAND devices store thenew data for the LBA on a new page, unlike hard disk drives (HDDs) thatcan rewrite individual physical sectors. Thus, a NAND device generallyrequires that a block be erased before new data can be written to theblock. Further, as described above, often a NAND device will write newdata for a given LBA to an erased page that is a different physical pagefrom the page previously used for that LBA. Thus, NAND devices alsogenerally require the device driver software or the separate controllerchip periodically initiate a process to erase data that is “stale” orout-of-date. As would be apparent to one of skill in the art, withoutperiodically erasing out-of-date data, the flash memory would fill upwith data that is mostly out-of-date. This inefficiency would reduce therealized flash memory capacity because less current data could bestored. Therefore, device driver software or controller chips generallyperiodically run a “garbage collection” routine adapted to provideefficient flash memory utilization by erasing out-of-date blocks. Anexample of a garbage collection routine is described in Kang. Garbagecollection routines impact performance of the flash memory system byutilizing processor resources and potentially delaying write operationsto the flash media.

However, NAND device blocks can be erased relatively few times beforedevice failure (typically on the order of 100,000 erasures). Therefore,over the operational life of an SSD, blocks of flash memory will failand become unusable. Thus, the device driver software or the separatecontroller chip should minimize the number of erasures, and must alsomaintain a record of bad blocks. For example, device driver software orcontroller chips might implement wear leveling to spread the erasing andwriting of blocks over the entire flash memory to avoid repeatedlyerasing and writing a given subset of blocks.

SUMMARY OF THE INVENTION

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

Described embodiments provide a method of transferring, by a mediacontroller, data associated with a host data transfer between a hostdevice and a storage media. A buffer layer module of the mediacontroller segments the host data transfer into one or more datatransfer segments. Each data transfer segment corresponds to at least aportion of the data. The buffer layer module allocates a number ofphysical buffers to a virtual circular buffer for buffering the one ormore data transfer segments. The buffer layer module transfers, by thevirtual circular buffer, each of the data transfer segments between thehost device and the storage media through the allocated physicalbuffers.

BRIEF DESCRIPTION OF THE DRAWINGS

Other aspects, features, and advantages of the present invention willbecome more fully apparent from the following detailed description, theappended claims, and the accompanying drawings in which like referencenumerals identify similar or identical elements.

FIG. 1 shows a block diagram of a flash memory storage systemimplementing logical-to-physical translation in accordance withexemplary embodiments of the present invention;

FIG. 2 shows an exemplary functional block diagram of processes employedby the flash memory storage system of FIG. 1;

FIG. 3 shows additional detail of the flash memory storage system ofFIG. 1;

FIG. 4 a shows an exemplary relation of a logical address of data to aphysical address of data as managed by a flash translation layer of theflash memory storage system of FIG. 1 operating in accordance withembodiments of the present invention;

FIG. 4 b shows an exemplary data structure for a summary page as mightbe employed by embodiments of the present invention;

FIG. 4 c shows an exemplary data structure for a Page Global Directory(PGD) as might be employed by embodiments of the present invention;

FIG. 4 d shows an exemplary data structure for an Active Block Table(ABT) as might be employed by embodiments of the present invention;

FIG. 5 a shows a flow diagram of a media read operation performed by abuffer layer of the flash memory storage system of FIG. 1 operating inaccordance with exemplary embodiments of the present invention;

FIG. 5 b shows a flow diagram of a media read operation performed at aflash translation layer of the flash memory storage system of FIG. 1operating in accordance with exemplary embodiments of the presentinvention;

FIG. 5 c shows a flow diagram of a media read operation performed by ahost layer of the flash memory storage system of FIG. 1 operating inaccordance with exemplary embodiments of the present invention;

FIG. 6 a shows a flow diagram of a media write operation performed by abuffer layer of the flash memory storage system of FIG. 1 operating inaccordance with exemplary embodiments of the present invention;

FIG. 6 b shows a flow diagram of a media write operation performed at aflash translation layer the flash memory storage system operating inaccordance with exemplary embodiments of the present invention;

FIG. 6 c shows a flow diagram of a media write operation performed by ahost layer of the flash memory storage system of FIG. 1 operating inaccordance with exemplary embodiments of the present invention;

FIG. 6 d shows a flow diagram of a summary page update routine inaccordance with exemplary embodiments of the present invention;

FIG. 7 a shows a block diagram of internal segmentation of large datatransfers employed by the flash memory storage system of FIG. 1;

FIG. 7 b shows a virtual circular buffer employed for media read andmedia write operations in accordance with exemplary embodiments of thepresent invention;

FIG. 8 shows a timing diagram of a media write operation employing thebuffer of FIG. 7 b;

FIG. 9 shows a flow diagram of a logical-to-physical translation tablereconstruction operation in accordance with exemplary embodiments of thepresent invention;

FIG. 10 a shows a flow diagram of a scan and group blocks sub-routine ofthe logical-to-physical translation table reconstruction operation ofthe exemplary embodiment of FIG. 9;

FIG. 10 b shows a block diagram of the flash media block groupingemployed by the scan and group blocks sub-routine of FIG. 10 a;

FIG. 11 shows a flow diagram of a process blocks and update datastructures sub-routine of the logical-to-physical translation tablereconstruction operation of the exemplary embodiment of FIG. 9;

FIG. 12 shows a flow diagram of a flexible cache allocation operation inaccordance with exemplary embodiments of the present invention;

FIG. 13 shows the size of the cache of the exemplary embodiment of FIG.12;

FIG. 14 shows a block diagram of a summary page cache data structure inaccordance with exemplary embodiments of the present invention;

FIG. 15 shows a diagram of a summary page cache in accordance withexemplary embodiments of the present invention;

FIG. 16 shows a block diagram of an operation to allocate a summary pagecache entry to an empty cache location in accordance with exemplaryembodiments of the present invention;

FIG. 17 shows a block diagram of an operation to update a pending cacheentry to a valid cache entry to in accordance with exemplary embodimentsof the present invention;

FIG. 18 shows a block diagram of an operation to update a pending cacheentry to a valid cache entry to in accordance with exemplary embodimentsof the present invention;

FIG. 19 shows a block diagram of an operation to abort a pending entryto the cache in accordance with exemplary embodiments of the presentinvention;

FIG. 20 shows a block diagram of an operation to invalidate a stalecache entry in accordance with exemplary embodiments of the presentinvention;

FIG. 21 shows a state transition diagram for a logical block address(LBA) stored in a cache in accordance with exemplary embodiments of thepresent invention;

FIG. 22 shows a flow diagram of a cache-media synchronization operationin accordance with exemplary embodiments of the present invention; and

FIG. 23 shows a data structure employed in for cache-mediasynchronization in accordance with exemplary embodiments of the presentinvention.

DETAILED DESCRIPTION

In accordance with embodiments of the present invention, a flashcontroller is provided that divides data transfers internally intosmaller segments (“chunks”) and employs one or more virtual circularbuffers to provide parallel processing of host-side and media-side datatransfers. A buffer layer module of the flash controller might allocateone or more physical buffers to the virtual circular buffer. Additionalphysical buffers might be allocated to the virtual circular bufferdepending on the availability of resources of the media controller. Theone or more physical buffers of the virtual circular buffer might beemployed to provide parallel processing of host-side and media-side datatransfers. Embodiments of the present invention might provide multiplevirtual circular buffers operating simultaneously to support parallelprocessing of multiple large data transfers across one or more storagedevices.

FIG. 1 shows a block diagram of flash memory storage system 100implementing a logical-to-physical translation in accordance withexemplary embodiments of the present invention. As shown, flash memorystorage system 100 is electrically coupled to communication link 102.Flash memory storage system 100 comprises flash controller 104, optionalexternal RAM buffer 114, and flash media 118. Although generallydescribed herein as flash media, media 118 might be implemented as atleast one of an SSD, an HDD, or a hybrid magnetic and solid statestorage system. Communication link 102 is employed for communicationwith one or more external devices, such as a computer system ornetworking device, which interface with flash memory storage system 100.Communication link 102 might be a custom-designed communication link, ormight conform to a standard communication protocol such as, for example,a Small Computer System Interface (“SCSI”) protocol bus, a SerialAttached SCSI (“SAS”) protocol bus, a Serial Advanced TechnologyAttachment (“SATA”) protocol bus, a Universal Serial Bus (“USB”), anEthernet link, an IEEE 802.11 link, an IEEE 802.15 link, and IEEE 802.16link, or any other similar interface link for connecting a peripheraldevice to a computer.

Flash controller 104 controls transfer of data between flash media 118and an external device coupled to communication link 102. Flashcontroller 104 might be implemented as a system-on-chip (SoC). Flashcontroller 104 might include internal RAM buffer 112 and might also becoupled to additional external memory, shown as external RAM buffer 114.In an exemplary embodiment, internal RAM buffer 112 comprises 128 kB ofstatic RAM (SRAM) and external RAM buffer 114 comprises 512 MB of doubledata rate version 2 dynamic RAM (DDR2 DRAM). RAM buffer 112 might act asa cache for processor 116, while RAM buffer 114 might act as aread/write buffer between flash media 118 and communication link 102.Processor 116 includes software and/or firmware as needed for operation,including for logical-to-physical translation in accordance withexemplary embodiments of the present invention, as describedsubsequently. Although shown in FIG. 1 as a single processor, processor116 might be implemented with multiple processors. For embodimentshaving multiple processors, inter-processor communication might beemployed, such as described in related U.S. patent application Ser. No.12/436,227.

FIG. 2 shows an exemplary functional block diagram of process modulesimplemented as software, hardware, or some combination thereof, withinprocessor 116 and flash controller 104. As shown in FIG. 2, host layer202 implements protocols to control flow of data between communicationslink 102 and flash controller 104. For example, host layer 202 mightprocess data access commands from communication link 102 and communicatewith flash translation layer (FTL) 208. In embodiments of the presentinvention, FTL 208 translates the logical-to-physical addresses (andvice-versa) of data stored in flash media 118, for example, by makingflash memory storage system 100 appear similar to a conventional HDD.FTL 208 temporarily stores data in a target buffer via buffer layer 210.In general, data transfers between flash media 118 and communicationlink 102 are buffered in the target buffer that includes at least one ofexternal RAM buffer 114 and internal RAM buffer 112. FTL 208 interfaceswith flash media 118 by flash low-level driver 212. Flash low-leveldriver 212 implements hardware-specific basic read and write operationsof flash memory 118, thus, separating the low-level hardware-specificsignal and timing requirements of the flash memory circuitry from thefunctionality of FTL 208. FTL 208 also enables garbage collection, errorrecovery, and wear leveling routines for flash media 118. Host layer202, buffer layer 210 and flash translation layer 208 might includeApplication Programming Interfaces (APIs), which are protocols orformats used by software to communicate between sub-applications withinthe software.

For example, flash controller 104 receives one or more requests forflash media access, such as read or write operations, from one or moreexternal devices via communication link 102. Such requests for access toflash media 118 generally include at least one logical block address(LBA) where data should be read or written. For example, the requestsmight be to read from or write to a i) single flash address, ii) a groupof contiguous flash addresses, or iii) a group of non-contiguous flashaddresses. Received requests are processed by host layer 202. Host layer202 i) controls host interface-specific commands (e.g. SATA commands),ii) coordinates host-side data transfers and command execution, and iii)processes any other host commands (e.g. status updates). Host layer 202is in communication with buffer layer 210. FTL 208 translates the LBAinto a physical address of the desired data. FTL 208 also interfaceswith buffer layer 210. Since data transfers between communication link102 and flash media 118 are temporally stored in buffer memory, bufferlayer 210 generally directs the data traffic between host layer 202 andFTL 208. For example, if an external host (not shown) provides, viacommunication link 102, data to be written to flash media 118, bufferlayer 210 might coordinate temporary storage of the data in buffer 114until FTL 208 coordinates writing the data to flash media 118.Similarly, if the external host requests to read data from flash media118, buffer layer 210 might temporarily store the data in buffer 114until host layer 202 coordinates sending the data to the host viacommunication link 102.

FIG. 3 shows an exemplary embodiment of flash media 118 and flashlow-level driver 212, in accordance with embodiments of presentinvention. As shown, flash media 118 might include one or more physicalsilicon dies, shown as flash dies 304(1) through 304(N). As shown, eachflash die is in communication with flash low-level driver 212 via a“lane”, shown as lanes 306(1) through 306(N). Additionally, flashlow-level driver 212 includes one or more lane controllers, shown aslane controllers 302(1) through 302(N), corresponding to each lane andflash die.

Embodiments of the present invention include groups of Superblockscalled wear-level units. Host requests might be striped across multiplewear-level units to provide parallel execution. Striping might beperformed on a per page basis, meaning that each page is striped acrossmultiple wear-level units. In exemplary embodiments of the presentinvention, a wear-level unit might correspond to one flash die as shownin FIG. 3. For example, flash dies 304(1) through 304(N) might beconfigured such that data is “striped” across two or more diesanalogously to hard drives in a redundant array of independent disks(RAID), with each die 304(1) through 304(N) corresponding to awear-level unit. Alternatively, embodiments of the present inventionmight configure each flash die 304(1) through 304(N) as a separate,stand-alone flash memory device without data striping.

FIGS. 4 a-4 d show block diagrams of exemplary data structures employedby FTL 208 for logical-to-physical translation of memory addresses. FIG.4 a shows an exemplary relation of a logical address of data (LBA 402)to a physical address of data (Superblock number 410, Block index 412and Page Index 414) as managed by FTL 208 of FIG. 2. FIG. 4 b showsSummary Page 420, FIG. 4 c shows Page Global Directory (PGD) 430, andFIG. 4 d shows Active Block Table (ABT 440). As described previouslywith regard to FIG. 2, when a host device requests access to flash media118, the request generally includes a logical block address (LBA), whichFTL 208 translates into a physical address of the desired data. As shownin FIG. 4 a, LBA 402 corresponds to at least one logical sector number(LSN) 404. LSN 404 corresponds to the smallest writable unit of a hostdevice coupled to communication link 102. For example, LSN 404 generallymight correspond to a sector size of 512 bytes, which is the typicalsector size for traditional hard drives (HDDs).

LSN 404 corresponds to a logical block number (LBN) 406 and a logicalpage number (LPN) 408. FTL 208 derives LBN 406 by dividing LSN 404 by anumber of sectors per block of flash media 118. FTL 208 derives LPN 408by dividing LSN 404 by a number of sectors per page of flash media 118.LBN 406 in turn corresponds to Superblock number 410 and block index412, while LPN 408 corresponds to page index 414. As described, aSuperblock generally is a logical collection of blocks representing afixed range of LBAs. FTL 208 derives Superblock number 410 and blockindex 412 from LBN 406 by dividing LBN 406 by a number of blocks perSuperblock, where Superblock number 410 corresponds to the quotient andblock index 412 corresponds to the remainder. Page index 414 is derivedfrom LPN 408 by dividing LPN 408 by a number of pages per block, andpage index 414 represents the physical page offset within the block. Forexample, if a flash page size is 4096 bytes, and the sector size is 512bytes, each flash page can store up to 8 sectors. An exemplary blockmight contain 128 pages. In this example, LPN 408 is equal to LSN 404divided by 8, and page index 414 is equal to LPN 408 divided by 128.

As described herein, each page includes a small spare area generallyused to store error correcting code (ECC) data. The ECC fields arewritten to the spare area by flash controller 104 (FIG. 1). In additionto storing ECC data, embodiments of the present invention might use thespare area of each page to store additional information forlogical-to-physical address translation. For example, FTL 208 of FIG. 2might store the wear-level unit number corresponding to the physicalblock in the spare area of one or more pages of the block. FTL 208 mightstore a sequence number (SN) in the spare area of one or more pages of aphysical block. The SN might represent the order in which FTL 208assigned the block to the Superblock. Each time a block is assigned fordata storage, the SN is incremented. Thus, blocks having a higher SNwere assigned more recently than blocks having a lower SN. The SN mightalso represent the order in which FTL 208 wrote the pages of the block,where every time a page is written, the SN is incremented such that morerecently written pages have a higher SN. FTL 208 might also store theLSN corresponding to the page in the spare area, or store a bad blockindicator (BBI) in the spare area of one or more pages of a block thathas failed (in whole or in part). Embodiments of the present inventionmight further utilize the spare area to support enterprise system sectorsizes (e.g. 520 or 528 bytes per sector instead of 512 bytes), such asdescribed in related U.S. patent applications Ser. Nos. 12/477,996 and12/478,013.

Each Superblock has a summary page, shown in FIG. 4 b as summary page420. Summary page 420 contains the summary mapping data for theassociated Superblock. For example, summary page 420 contains the blockindex and page index, shown as block indices 422(1)-422(X) and pageindices 424(1)-424(Y) for all X blocks and all Y pages in Superblockassociated with the summary page. The summary page block indices includeall physical blocks (both data blocks and update blocks) within theSuperblock. Together, block indices 422(1)-422(X) and page indices424(1)-424(Y) are stored as data pointer 423, which points to thephysical address of each page of the Superblock. Summary page 420 mightalso include a pointer to the location of the Active Block (active blockpointer 425) and next free page (free page pointer 426) within theSuperblock, as well as the location of the next page of a partiallywritten block as a result of garbage collection (partial block pagepointer 427). The summary page generally includes all necessaryinformation to convert a logical address to a physical address of flashmedia 118. Embodiments of the present invention might perform garbagecollection to erase pages containing out-of-date data, such as describedin related U.S. patent application Ser. No. 12/508,879. As will bedescribed subsequently with regard to FIGS. 6 b-d, the summary page isupdated periodically by FTL 208 to include more up-to-date mapping datathat might be stored in ABT 440 or PGD 430 for each Superblock, forexample, the block index of the active block and the page index to thenext free page.

As shown in FIG. 4 c, PGD 430 is a data structure that includes a seriesof entries for each Superblock (shown as Superblocks 0-Q) in awear-level unit. PGD 430 might include the pointer (block index and pageindex) to the summary page associated with each Superblock (for example,summary page pointer 434 corresponding to Superblock 1 432). PGD 430might include ABT pointer 436 that points to the location of the activeblock table (e.g. ABT 440) for the Superblock. PGD 430 might be storedin a reserved area of flash media 118 with other mapping data, such assummary pages.

Each Superblock has an Active Block Table (ABT), shown in FIG. 4 d asABT 440. ABT 440 tracks the currently active block in each Superblock ina wear-level unit. ABT 440 contains, for each Superblock Q in awear-level unit, a list of page offsets indicating the written pages ofthe active block in the Superblock, shown as page offsets 442(0)-442(N).ABT 440 might be written in top-down order such that page offset 442(0)corresponds to the first page written in the active block, and pageoffset 442(N) corresponds to the most recently written page in theactive block. As will be described herein, ABT 440 might representmapping data for the active block of a Superblock for write operationsthat have been completed to flash, but the corresponding summary pageshave not yet been updated. ABT 440 might be stored in RAM (e.g. at leastone of buffer 112 and buffer 114) and reconstructed at startup of thestorage device from summary pages stored in media 118. The interactionbetween PGD 430, ABT 440 and updating of summary pages (e.g. summarypage 420 of FIG. 4 b) will be described in greater detail with regard toFIG. 6 d. As shown in FIG. 4 d, each wear-level unit might also containa list of failed blocks (bad block list 428) that includes pointers (badblock pointer 429) to failed blocks within the wear-level unit.

In exemplary embodiments of the present invention, summary pages (e gsummary page 420 of FIG. 4 b) for all the Superblocks of a wear-levelunit are stored “out-of-line” in a separate Superblock (the summary pageSuperblock). Thus, no pages in data Superblocks are used to storemapping data, keeping the number of available pages per block to a powerof two. In exemplary embodiments of the present invention, one or moreSuperblocks of each wear-level unit (the “map Superblocks”) might bereserved to store mapping data. The summary page of the map Superblock(the “map page”) and is saved “in-line” as the first page of the mapSuperblock. FTL 208 (FIG. 2) is configured to skip reserved Superblocks,making them inaccessible by host requests, thus “reserving” theSuperblocks for mapping data.

FIG. 5 a shows a system-level flow diagram of media read operation 500performed generally by buffer layer 210 of flash memory storage system100 of FIG. 1. As shown in FIG. 5 a, media read operation 500 might be arequest to read one or more contiguous addresses of flash media 118.Read requests for one or more non-contiguous addresses of flash media118 might be processed substantially the same as shown in FIG. 5 a, or,alternatively, read requests for non-contiguous addresses might beprocessed as described in related U.S. patent application Ser. No.12/508,915. As shown in FIG. 5 a, buffer layer 210 responds to a readrequest from host layer 202 at step 502. At step 504, buffer layer 210might segment the read into smaller internal data transfers (“chunks”).Each chunk corresponds to a predefined number of LBAs (“sectors”). Astarting LBA is retained with each chunk to identify the sectorscorresponding to the chunk. A chunk boundary exists between the last LBAof one chunk and the first LBA of the next chunk. Embodiments of thepresent invention employ a chunk size that is substantially equal to thepage size of flash media 118 (for example, 2^(N) bytes, where N is aninteger). Thus, for example, a read operation might include multiplecontiguous chunks (e.g. chunks having contiguous LBAs).

At step 506, buffer layer 210 allocates buffer space for one or morechunks in the current segment of the read operation for which data is tobe read. Buffer layer 210 might allocate buffer space for the entireread and transfers all of the chunks from flash media 118. At step 508,buffer layer 210 requests data from FTL 208, corresponding to at least aportion of the data requested by the read request received at step 502,to be transferred from flash media 118. At step 510, FTL 208 providesthe chunk data to buffer layer 210 and, at step 512, buffer layer 210temporarily stores the data in buffer 114. At step 514, buffer layer 210requests that host layer 202 retrieve the chunk data stored in buffer114 at step 512. At step 516, host layer 202 transfers the chunk data tocommunication link 102. At step 518, buffer layer 210 deallocates thespace in buffer 114 that was allocated in step 506 for the current groupof one or more chunks. At step 520, if there are more chunks totransfer, processing returns to step 506 for buffer layer 210 toallocate buffer space for the next group of one or more chunks to beprocessed. If there are no more chunks to be transferred, processingcontinues to step 522, where the read operation ends.

As will be described in greater detail with regard to FIG. 7 a, FIG. 7 band FIG. 8, embodiments of the present invention might perform host-sideoperations, for example steps 514 and 516 of FIG. 5 a, for a first groupof one or more chunks, and media-side operations, for example steps508-512 of FIG. 5 a, for a subsequent group of one or more chunks, inparallel. For example, by segmenting the read operation into chunks, afirst set of chunks might be transferred between FTL 208 and bufferlayer 210 (step 510), and host layer 202 might then transfer the firstset of chunks to communication link 102 (step 516). Concurrently withone or more of the host-side operations for the first set of chunks, asecond set of chunks for the same read operation might be transferredfrom FTL 208 to buffer layer 210 (step 510), and so on, until all chunksfor the read operation are transferred. Thus, embodiments of the presentinvention provide the ability to perform host side and media sidetransfers in parallel.

FIG. 5 b shows a flow diagram of an exemplary flash media read operation530 executed by FTL 208 (e.g. the media-side read operations at steps508-512 of FIG. 5 a). At step 532, the read operation is initiated, forexample, in response to a flash media read request received from anexternal device coupled to communication link 102, as described withregard to FIGS. 1, 2 and 5 a. As described with regard to FIG. 4 a, theread request includes a logical block address (LBA) that FTL 208translates into an actual address of the desired data at step 534. Thus,at step 534, FTL 208 determines the Superblock number, Block index andPage index of the first page to be read. At step 536, FTL 208 reads thePage Global Directory (e.g. PGD 430 of FIG. 4 c) to read the ActiveBlock Table pointer for the requested Superblock. At step 538, FTL 208scans the ABT (e.g. ABT 440 of FIG. 4 d) of the requested Superblock, inreverse order from most recent write to the first write of the activeblock, to determine if the requested page pointer is stored in the ABT(i.e. if the page was in the active block). At step 540, if the pagepointer is stored in ABT 440, the page pointer is then read from the ABTat step 541, and the requested page is read from flash at step 554. Ifthe page pointer is not in ABT 440, FTL 208 locates the summary page (eg summary page 420 of FIG. 4 b) for the requested Superblock at step542.

If, at step 544, no summary page exists for the requested Superblock, anerror occurred and an error code or other predetermined data might bereturned at step 552. If, at step 544, the summary page for therequested Superblock exists, at step 546 the summary page is read. Thesummary page can either be read from flash, or as described herein, froma cached copy of the summary page stored in RAM. At step 548, thepointer for the requested page is read from the summary page based onthe Block Index and Page Index. At step 550, if the requested pagepointer does not exist in the summary page, an error code or otherpredetermined data might be returned at step 552. At step 550, if therequested page pointer exists in the summary page, at step 554 FTL 208reads the requested page from flash media 118, as described with regardto FIG. 2. As indicated by dashed lines 555 and 557, respectively, andas will be described in greater detail with regard to FIG. 5 c, ifadditional pages remain to be read from flash, the process returns tostep 534, otherwise, the read operation ends at step 556.

FIG. 5 c shows a flow diagram of an exemplary flash media read operation570 executed by host layer 202. As described previously with respect toFIG. 2, a host read request might request to read data from i) a singleflash address (page), ii) multiple, sequential flash pages, or iii)multiple, non-sequential flash pages. As shown in FIG. 5 c, at step 571,a host read operation is started. At step 572, a test determines whetherthe read operation is for multiple non-sequential pages (or a singlepage), or multiple sequential pages. If, at step 572, the read operationis for multiple non-sequential pages (or a single page), at step 574,host layer 202 requests that FTL 208 initiate media read operation 530shown in FIG. 5 b. At step 576, if the final page of the read operationwas read, the read operation proceeds to step 586 and ends. Otherwise,if step 576 determines that the final page was not read, indicating thatthe read operation has non-sequential pages remaining to be read, atstep 578, the next LBA is retrieved, and processing returns to step 574to read the next address. If, at step 572, the read operation is formultiple sequential pages, then, at step 580, host layer 202 requeststhat FTL 208 initiate media read operation 530 shown in FIG. 5 b. Atstep 582, if there are additional pages of the read request remaining tobe read, the page offset is incremented at step 584 and the next page isread by FTL 208 at step 580, and so on, until the last requested pagehas been read. At step 582, if the last requested page has been read, atstep 586 the read operation ends.

Thus, as shown in FIGS. 5 b and 5 c, for a read operation, at most, twoflash media read operations occur per each requested address: i) if thesummary page data is not cached in RAM, the summary page is read fromflash (e.g. step 546), and ii) the actual data location is read fromflash (e.g. step 554). For a sequential read operation, this process issimplified. For the first page of a sequential read, at most, two flashmedia read operations occur per address: i) if the summary page is notcached in RAM, the summary page is read from flash (e.g. step 546), andii) the actual data location is read from flash (e.g. step 554). Forsubsequent pages of the sequential read operation, the page addressmight simply be incremented (e.g. step 584) to read the next page fromflash (e.g. step 580).

FIG. 6 a shows a flow diagram of a media write operation performedgenerally by buffer layer 210 of flash memory storage system 100 ofFIG. 1. As shown in FIG. 6 a, media write operation 600 might be arequest to write one or more contiguous addresses of flash media 118.Write requests for one or more non-contiguous addresses of flash media118 might be processed substantially the same as shown in FIG. 6 a, or,alternatively, write requests for non-contiguous addresses might beprocessed as described in related U.S. patent application Ser. No.12/508,915. As shown in FIG. 6 a, at step 602, host layer 202 (FIG. 2)responds to a write request from communication link 102. At step 604,buffer layer 210 groups the sectors of the write operation into chunksbased on the starting LBA. At step 606, buffer layer 210 allocatesbuffer space for chunks of the write operation being processed. At step608, buffer layer 210 requests the data for the current chunks from hostlayer 202. At step 610, host layer 202 transfers the chunk data tobuffer layer 210, and, at step 612, buffer layer 210 stores the chunkdata in buffer 114.

At step 614, buffer layer 210 provides data for the one or more chunksto FTL 208. At step 616, FTL 208 writes one or more pages of the chunkdata to flash media 118. At step 618, buffer layer 210 deallocates thespace in buffer 114 allocated at step 606 for the current chunks. Atstep 620, if there are additional chunks having data to be written,processing returns to step 606. If there are no additional chunks to bewritten, at step 622, the write operation is ended. As described abovewith regard to the read operation of FIG. 5 a, and as will be describedin greater detail with regard to FIG. 7 a, FIG. 7 b and FIG. 8,embodiments of the present invention might perform host-side operations,for example steps 608-612 of FIG. 6 a, for a first group of one or morechunks, and media-side operations, for example steps 616 and 618 of FIG.6 a, for a subsequent group of one or more chunks, in parallel.

FIG. 6 b shows a flow diagram of an exemplary flash media writeoperation 630 executed by FTL 208 (e.g. the media-side write operationsat step 616 of FIG. 6 a). At step 632, a write operation is initiated,for example, in response to a flash media write request received from anexternal device coupled to communication link 102. As described withregard to FIG. 4, the write request includes a logical block address(LBA) that FTL 208 translates into an actual address of the data at step634. Thus, at step 634, FTL 208 determines the Superblock number, Blockindex and Page index of the first page to be written. At step 636, FTL208 determines if an Active Block exists for the requested Superblock byscanning the PGD (e.g. PGD 430 of FIG. 4 c). If an Active Block does notexist, such as, for example, if there is no entry for the Superblock inthe PGD, then at step 638 a new active block is allocated for theSuperblock from a pool of free blocks within the wear-level unit. Atstep 640, the Active Block index and Active Page index are stored to thePGD entry associated with the requested Superblock. If an Active Blockdoes exist, at step 642, the block index and the page index of theActive Block are retrieved from the PGD. Once the block index and pageindex of the Active Page are determined, the requested data is writtenat the active page address at step 644.

At step 646, if the Active Block is not full (or if the number ofwritten pages in the Active Block is below a threshold), then, at step658, the active page index is updated to point to the next free page inthe active block and is stored to the PGD. If, at step 646, the ActiveBlock is full (or if the number of written pages in the Active Block isabove a threshold), a new active block might be allocated and theprocess advances to step 648. At step 648, the summary page for theSuperblock containing the active block is read. The summary page caneither be read from flash, or as described herein, from a cached copy ofthe summary page stored in RAM. If a summary page for the Superblockcontaining a newly allocated active block does not exist, a new summarypage is allocated. At step 650, the data from the summary page and theactive block table is merged to create an updated summary page. FTL 208allocates a new active block for the Superblock at step 652 and writes anew summary page for the Superblock to flash at step 654. At step 656,FTL 208 updates the Page Global Directory (e.g. PGD 430 of FIG. 4 c) topoint to the new summary page and the new active block for theSuperblock. Then, at step 658, the written page offset is stored in ABT440, and the current active page and the active block are stored in PGD430. The next active page is the current page incremented by one, orpage 0 of the next sequential block.

As indicated by dashed line 657, steps 646 through 658 could be repeatedif the Active Block of the summary page superblock also happened tobecome full at the same time. For example, a write operation occurs andthe active block is full, as described previously. Upon updating thesummary page at step 654, the active block of the summary pagesuperblock could become full. In that instance, steps 646 through 658would be repeated to allocate a new active block for the summary pagesuperblock. Otherwise, as indicated by dashed line 661, the media writeoperation ends at step 660.

FIG. 6 c shows a flow diagram of host write operation 663 executed byhost layer 202. As described herein, a host write request might requestto write data to i) a single flash page, ii) multiple, sequential flashpages, or iii) multiple, non-sequential flash pages. As shown in FIG. 6c, at step 664 a host write operation is started. At step 665, adetermination is made as to whether the write operation is for multiplenon-sequential pages (or a single page), or multiple sequential pages.If, at step 665, the write operation is for multiple non-sequentialpages (or a single page), then, at step 666, host layer 202 requeststhat FTL 208 initiate media write operation 630 shown in FIG. 6 b. Aftereach page is written, at step 668, if it is determined that the finalpage of the write operation has been written, the host write operationends at step 675. If, at step 668, the write operation hasnon-sequential pages remaining to be written, then, at step 670, thenext LBA is retrieved, and processing returns to step 666 to write thenext address.

If, at step 665, the write operation is for multiple sequential pages,then, at step 671 host layer 202 requests that FTL 208 initiate mediawrite operation 630 shown in FIG. 6 b. At step 672, if there areadditional pages of the write request remaining to be written, the pageoffset is incremented at step 673 and the process returns to step 671 towrite the next page, and so on, until the last page has been written. Atstep 672, if the last page was written, at step 675 the write operationends.

As shown in FIGS. 6 b and 6 c, in general write operations require onlya single flash operation (writing to the active block). The active blockdata might be recovered from ABT 440, which is stored in RAM. If theactive block being written becomes full, additional steps for updatingthe mapping data (e.g. step 667 or step 674) might require flash mediaaccesses, for example, to: i) read the summary page from flash (e.g.step 648), ii) write a new summary page to flash (e.g. step 654), iii)update the page global data for the superblock (e.g. step 656).

In embodiments of the present invention, the summary pages for eachSuperblock might be periodically updated, for example, during idle timeof flash memory storage system 100. As described with regard to FIG. 4d, ABT 440 might be employed to buffer mapping data for flash mediawrite operations that are completed between updates of the summarypages. FIG. 6 d shows a flow diagram of exemplary summary page updateroutine 680 executed by FTL 208. Summary page update routine 680 mightbe performed when ABT 440 becomes full (or is filled more than apredetermined threshold). At step 682, update summary page routine 680is initiated by FTL 208. At step 684, FTL 208 reads the Page Global Data(e.g. PGD 430 of FIG. 4 c) and the Active Block Table (e.g. ABT 440 ofFIG. 4 d). ABT 440 might be written in one order (e.g. top-down) andread in the opposite order (e.g. bottom-up), generally forming alast-in, first-out (LIFO) buffer. By reading ABT 440 in the oppositeorder it is written, in the event that a certain block is writtenmultiple times before the summary page is updated, FTL 208 only updatesthe summary page once for every full block, avoiding multiple updates ofthe same summary page for stale data. FTL 208 might scan the entries ofABT 440, for example, sequentially from the highest indexed Superblockto the lowest indexed Superblock, to determine if ABT 440 contains datafor one or more Superblocks that are more recent than summary page(s)for the Superblock(s). PGD 430 is read by Superblock number.

At step 688, FTL 208 merges the entries of ABT 440, PGD 430 and summarypage 420 for any Superblocks determined to have summary pages that areout-of-date. By merging the ABT entries, PGD entries and summary pageentries, FTL 208 creates a new, up-to-date summary page for theSuperblock(s). At step 690, a new active block is allocated, and theactive block pointer in the summary page (e.g. pointer 425) is updated.At step 691, the new summary page is written. At step 692, the map page(i.e. the summary page for the map Superblock) is updated to include thenew page addresses for the summary page and the summary page pointer andactive block pointer are updated in PGD 430. At step 694, all mappingdata has been updated and summary page update routine 680 is ended.

The frequency with which FTL 208 performs periodic summary page updateroutine 680 is generally a tradeoff between the number of writeoperations to flash media 118 and the amount of RAM (e.g. buffer 114)needed to store ABT 440. The more frequently the summary pages areupdated, the more write operations are performed and the less RAM isneeded to store ABT 440. The less frequently the summary pages areupdated, the fewer write operations are performed and the more RAM isrequired to store ABT 440. The fewer write operations are performed, thefewer erase operations are performed, potentially extending theoperating life flash media 118, but requiring more RAM. Embodiments ofthe present invention provide that the summary page update frequencymight be a user selectable setting of flash memory controller 104.Alternatively, at system startup, flash memory controller 104 mightautomatically detect the amount of RAM available (for example, the sizeof buffer 114) and configure ABT 440 to a default size.

Although an HDD controller might generally access a single HDD serially,an SSD controller, such as flash controller 104 of FIG. 1, might accessone or more flash devices in parallel, shown in FIG. 3 as flash dies304(1)-304(N). In some instances, large data transfers might spanmultiple of the flash dies 304(1)-304(N). Embodiments of the presentinvention divide data transfers internally into smaller segments(“chunks”) and employ one or more virtual circular buffers to facilitateparallel processing of host-side and media-side data transfers.

FIG. 7 a shows an exemplary data transfer, 702, for 1 MB of data. Datatransfer 702 might be a host-side data transfer (e.g. a flash writeoperation) of data to be written from a device coupled to communicationlink 102 (FIG. 1) to flash media 118, or data transfer 702 might be amedia-side data transfer (e.g. a flash read operation) of data read fromflash media 118 to be provided to one or more devices coupled tocommunication link 102. For data transfers larger than a predeterminedthreshold, buffer layer 210 (FIG. 2) might segment host-side mediarequest 702 into smaller into smaller internal data transfers. For theexample of FIG. 7 a, buffer layer 210 might split data transfer 702 intofour smaller data transfers shown as chunks 704(1)-704(4). As shown inthe example of FIG. 7 a, the predetermined threshold is 256 kB, thus,data transfer 702 is divided into four 256 kB chunks, but otherthreshold values might be employed. As shown, each of chunks704(1)-704(4) corresponds to 256 kB segments 706(1)-706(4) of datatransfer 702. The maximum size of the chunks is determined by the sizeof the physical buffers, shown in FIG. 7 b.

FIG. 7 b shows exemplary virtual circular buffer 700. Virtual circularbuffer 700 might be controlled by buffer layer 210 (FIG. 2), and mightbe located in at least one of internal RAM buffer 112 and external RAMbuffer 114 (FIG. 1). As shown, virtual circular buffer 700 might includetwo physical buffers, shown as buffer1 710 and buffer2 712. Inembodiments of the present invention, the number of physical buffersemployed by virtual circular buffer 700 might be selectable. Forexample, if flash controller 104 is under relatively low workload forlarge data transfers, buffer layer 210 might allocate an additionalphysical buffer (for example, a “buffer3”) to virtual circular buffer700. The advantage of allocating an additional physical buffer is higherutilization of the buffer hardware (FIG. 1) and software engines (FIG.2). Buffer1 710 and buffer2 712 are configured to temporarily store datachunks 704(1)-704(4) as described subsequently.

FIG. 8 shows a timing diagram of a write operation employing virtualcircular buffer 700. As host transfer 802 provides chunk1 704(1) tobuffer1 710 at time1 803, media transfer 804 is queued for chunk1704(1). At time2 806, when the host transfer for chunk1 704(1) iscomplete, media transfer 804 starts providing chunk1 704(1) to flashmedia 118 (FIG. 1). In parallel with media transfer 804, host transfer805 for the next chunk, chunk2 704(2), is provided to buffer2 712. Whenmedia transfer 804 is complete, buffer1 710 is freed to receive the nextchunk, chunk3 704(3), by host transfer 808 at time3 809. In parallelwith host transfer 808, once host transfer 805 is complete, mediatransfer 807 provides chunk2 704(2) to flash media 118, and so on, untilall chunks of the data transfer are processed.

In some embodiments of the present invention, a physical buffer (e.g.buffer1 710 and buffer2 712) is reused within virtual circular buffer700 as soon as the buffered data is transferred to its destination (forexample, flash media 118 in the example of FIG. 8). This minimizes theeffect of large data transfers on the buffer space available in buffers112 and 114 for other operations of flash controller 104. Alternatively,flash controller 104 might be configured to replace the physical buffersof virtual circular buffer 700 with alternate physical buffers inbetween handling of chunks for a large data transfer. This might allowbuffer layer 210 flexibility in configuring and allocating buffer spacesuch as, for example, selectably increasing or decreasing the number ofphysical buffers for a virtual circular buffer, as described with regardto FIG. 7 b.

Embodiments of the present invention provide multiple virtual circularbuffers (e.g. virtual circular buffer 700 of FIG. 7 b) operatingsimultaneously to support parallel processing of multiple large datatransfers. For example, referring back to FIG. 3, buffer layer 210employing N virtual circular buffers allows processing of multiple largedata transfers in parallel because data is transferred in parallelbetween the N virtual circular buffers and the N flash dies304(1)-304(N). Further, the number of virtual circular buffers inoperation might be selectable by buffer layer 210. For example, if flashcontroller 104 is under a heavy workload for large data transfers,buffer layer 210 might allocate an additional virtual circular buffer toprovide parallel processing of the large data transfers. Virtualcircular buffers are useful for skip-read and skip-write operations,such as described in related U.S. patent application Ser. No.12/508,915. Virtual circular buffers are also useful in performing datatransfers across logical partition boundaries (e.g. Superblockboundaries).

On startup of flash memory storage system 100, mapping data stored involatile memory (e.g. RAM buffers 112 and 114) requires reconstruction.The reconstruction process is desirably completed quickly to allowaccess of flash media 118. For example, ABT 440 is stored in RAM, and isreconstructed on startup to allow access of flash media 118. FIG. 9shows a flow diagram of map data reconstruction 900. At step 902, FTL208 initiates reconstruction 900, for example, on startup of flashmemory storage system 100. At step 903, FTL 208 requests that bufferlayer 210 allocate space in RAM (e.g. at least one of buffer 112 andbuffer 114 of FIG. 1) for ABT 440, which is initialized to predetermineddefault values. At step 904, FTL 208 scans the blocks within eachSuperblock of each wear-level unit and groups the blocks based on blocktype. Step 904 will be described in greater detail with regard to FIG.10 a. At step 906, FTL 208 processes the grouped blocks and updates thecorresponding mapping data structures (e.g. the data structures of FIGS.4 a-d). Step 906 will be described in greater detail with regard to FIG.11. At step 908, if additional blocks remain to be scanned andprocessed, processing continues to step 910 where the block index isincremented and the next block is scanned at step 904. This continuesuntil, at step 908, FTL 208 determines that all the blocks of flashmedia 118 have been scanned and processed, in which case, processingadvances to step 912. At step 912, FTL 208 determines if any blocks didnot appear the ABT, the summary pages, or the bad block list. For thedescribed embodiment, blocks that did not appear in these datastructures are presumed to have been in queue for erasure at the lastpower down. Thus, at step 914, these blocks are again placed in thequeue for erasure. If, at step 912, no blocks need to be queued forerasure or, after step 914 when blocks are placed in the queue,processing continues to step 916 where the reconstruction operation iscomplete.

FIG. 10 a shows an exemplary flow diagram of step 904 of thereconstruction process of FIG. 9, which performs the scan and groupblocks sub-routine. In general, there are six possible categories forblocks to be grouped into: i) anchor blocks, ii) map blocks, iii)summary blocks, iv) data blocks, v) free blocks and vi) bad blocks. Atstep 1002, scan and group blocks sub-routine 904 is initiated. At step1004, FTL 208 reads metadata that is stored in the spare area of thefirst page of the block. As described herein, this metadata mightinclude the host LBA and media LSN corresponding to the block, thewear-level unit number corresponding to the block, a block type of theblock, and the sequence number corresponding to the block. At step 1006,if the read of metadata at step 1004 is unsuccessful, processingcontinues to step 1008.

At step 1008, FTL 208 erases the block. At step 1010, if the erase ofthe block was successful, the block is then added to the free block listat step 1014. As described herein, blocks in the free block list mightbe allocated by FTL 208 as Update Blocks to a Superblock when additionaldata blocks are required to support write operations. If the erase ofthe block was unsuccessful, the block cannot be erased and has failed.In general, with flash memory devices, after a successful eraseoperation, all the bits of the block are set to logic 1. A failed eraseoperation might be detected if the block is read and one or more bitswithin the block are not set to logic 1. At step 1012, if the erase ofthe block was unsuccessful, the block address is added to the bad blocklist (e.g. bad block list 428 of FIG. 4 d) corresponding to thewear-level unit containing the Superblock. Additionally, FTL 208 mightattempt to write a bad block indicator flag in the spare area of one ormore pages of the failed block. After the bad block list is updated,processing continues to step 1030 where the scan and group blockssub-routine is ended and processing returns to step 906 of FIG. 9.

At step 1006, if the read of metadata at step 1004 is successful,processing continues to step 1016. At step 1016, the host LBA and themedia LSN of the block are determined, for example, from the metadataread at step 1004. At step 1018, if the host LBA of the block is greaterthan or equal to 0, then the process continues to step 1028 where theblock is determined to be a data block. If, at step 1018, the host LBAis not greater than or equal to 0, then the block might be an anchorblock, a summary block, or the map block.

For example, as shown in FIG. 10 b, flash media 118 might be dividedinto one or more separate physical spaces, shown as anchor space 1052and data space 1054. Anchor space 1052 contains data that must be storedin particular physical blocks (anchor blocks), thus, the blocks are“anchored” in a particular physical position in flash media 118. Anchorblocks might store at least a portion of the software or firmware forflash controller 104, or might store configuration files or other datarequired by flash controller 104 at power up. As described herein, thefirst block (block 0) of a flash die is generally provided from themanufacturer error-free, and might generally be used as an anchor block.Data space 1054 holds all other data, including user data (data blocks)and mapping data. Mapping data, such as the map block and summaryblocks, might be stored in reserved space 1056, which is one or moresegments of data space 1054 that are reserved for storing mapping data.Reserved space 1056 is not accessible by host requests (e.g. host readand write requests). In exemplary embodiments of the present invention,reserved space 1056 is placed immediately after anchor space 1052, or atthe end of data space 1054. Since they are not accessible by the host,blocks in anchor space 1052 and reserved space 1056 generally might nothave corresponding host LBAs.

Referring back to FIG. 10 a, if, at step 1018, the host LBA was notgreater than or equal to 0, then at step 1021, if the LSN is equal to 0,the block is determined to be a map block at step 1023. The map block isthe location of the map page (i.e. the block reserved for storing thesummary page of the summary page Superblock, as described with regard toFIG. 4 d). As described herein, in some embodiments of the presentinvention, the map page might be stored in the first block after theanchor blocks such that the map page can always be located by FTL 208,even if mapping data has been corrupted. If the LSN is greater than 0,processing continues to step 1022, where, if the LSN is equal to an LSNin the reserved Superblock(s), then at step 1024 the block is determinedto be a summary block (i.e. a block reserved for storing summary pagesof data Superblocks).

If, at step 1022, the LSN was not equal to a reserved LSN, at step 1023the LSN of the block is checked against the LSNs of anchor space 1052.If, at step 1023 the LSN of the block is equal to an LSN in the anchorspace, at step 1026, the block is determined to be an anchor block. If,at step 1023, the LSN was not recognized by FTL 208, at step 1027, anerror code might be generated and flash controller 104 might performsubsequent processing. Once the block type is determined, for example,by one of steps 1012 (bad block), 1014 (free block), 1020 (anchorblock), 1023 (map block), 1024 (summary block), and 1028 (data block),processing continues to step 1030, where scan and group blockssub-routine 904 is ended and processing returns to step 906 of FIG. 9.

Alternative embodiments of the present invention might simplify theblock type determination. As described herein, metadata might be storedin the spare area of the first page of each block. Exemplary embodimentsof the present invention might store a block type field in the metadata.This block type field might include a code to indicate that thecorresponding block is one of i) a bad block, ii) an anchor block, iii)a reserved block, iv) a summary block, v) a map block, vi) a data block,and vii) an erased block. This block type metadata field might be storedeach time a block is written. For example, flash memory storage system100 might be initially programmed with firmware during a manufacturingprocess. During this initial programming, as blocks used to storeelements of the firmware are written, the corresponding block type fieldmight be written to indicate that these blocks are anchor blocks. Duringinitial programming, one or more reserved areas of media 118 might bedetermined, and the block type field for these blocks might be set toindicate that the blocks are reserved. Similarly, during initialprogramming, if any bad blocks are detected, the corresponding blocktype field might be set to indicate that the block is bad. After initialprogramming during manufacturing, the block type field for all otherblocks might be set to indicate that the blocks are erased. These erasedblocks are available for subsequent use by flash memory storage system100, and as each block is written as summary blocks or data blocks, oras each block is subsequently erased, the corresponding block type fieldmight be updated accordingly.

FIG. 11 shows a flow diagram of the process blocks and update datastructures sub-routine performed at step 906 of the reconstructionprocess of FIG. 9. In embodiments of the present invention, processblocks and update data structures sub-routine 906 might not be performeduntil the block types of all blocks have been determined At step 1102,process blocks and update data structures sub-routine 906 is initiated.At step 1104, if the block being processed by FTL 208 was determined tobe a summary block in step 904 of FIG. 9, processing continues to step1106. At step 1106, the first page of the summary block is scanned todetermine the sequence number associated with the page and theSuperblock associated with the page. As described herein, the sequencenumber might represent the order in which FTL 208 allocated blocks tothe Superblock and wrote the pages of the block. At step 1108, thesequence number of the block is compared to the sequence number of theactive block stored in Page Global Directory (PGD) 430 that was createdat step 903 of FIG. 9 and is initialized to predetermined defaultvalues. If the sequence number of the summary page stored in PGD 430 isgreater than or equal to the sequence number of the page read at step1106, at step 1110 PGD 430 is up-to-date, and the process continues tostep 1114. If, at step 1108, the sequence number of the summary pagestored in PGD 430 is less than the sequence number of the page read atstep 1106, at step 1112 PGD 430 is updated to point to the page index ofthe more recently written summary page data, and the process continuesto step 1114. At step 1114, if the last written page of the currentblock has not been scanned, at step 1116 the page index is incrementedand the process returns to step 1106 to scan the next page. The processcontinues until the last written page has been scanned. At step 1114, ifthe last written page of the current block has been scanned, at step1142, the process returns to step 908 of FIG. 9.

If, at step 1104, the block is not a summary block, processing continuesto step 1118. At step 1118, if the block being processed by FTL 208 instep 904 of FIG. 9 is a data block, the process continues to step 1120.At step 1120, if the current block is not the active block of theSuperblock, processing might continue to step 1106, and the PGD might beupdated similarly as described for a summary block. At step 1120, if thecurrent block is the active block of the Superblock, the processcontinues to step 1122. At step 1122, the active block is scannedsequentially to find the last written page of the active block. At step1124, the page offsets stored in ABT 440 are updated to reflect theorder of writes to the active block, at step 1126, the page offsetsstored in ABT 440 are up-to-date, and the process continues to step1130.

At step 1130, FTL 208 checks to see if the active block is full, or ifthe amount free pages left in the active block has reached a minimumthreshold. At step 1130, if the Active Block is full (or if the numberof written pages in the Active Block is above a threshold), a new activeblock is allocated at step 1132, similarly as described with regard toFIG. 6 b. At step 1134, FTL 208 updates the summary page (e.g. summarypage 420 of FIG. 4 b) for the Superblock associated with the activeblock, similarly as described with regard to FIGS. 6 b and 6 d. At step1136, FTL 208 updates the active block table (e.g. ABT 440 of FIG. 4 d)to point to the new active block allocated at step 1132 and also updatesPGD 430 (FIG. 4 c) such that ABT pointer 436 points to the new activeblock allocated at step 1132. Then, at step 1142, sub-routine 906 endsand the process returns to step 908 of FIG. 9. At step 1130, if theactive block is not full, the process continues to step 1142, wheresub-routine 906 ends and the process returns to step 908 of FIG. 9.

At step 1138, if the block being processed by FTL 208 in step 904 ofFIG. 9 is the map block, the process continues to step 1140. At step1140, FTL 208 locates the last written page of the map block to locatethe most recent map page in order to locate the most recent location ofthe summary pages for each Superblock. Once the last written page of themap block is located at step 1140, PGD 430 is updated to point to thecurrent location of the map page at step 1141. If, at step 1138, theblock is either a free block or a bad block, the sub-routine of step 906ends at step 1142, where the process returns to step 908 of FIG. 9.

Embodiments of the present invention provide for at least one of RAMbuffer 112 and RAM buffer 114 to act as a read/write data cache for databeing transferred between media 118 and communication link 102. Ingeneral, an efficient hash table might be configured such that thememory allocated to it is approximately double the number entriesexpected to be stored in the hash table because there are diminishingreturns for maintaining a larger hash table with a limited number ofentries. Further, as described herein, embodiments of the presentinvention might configure RAM buffers 112 and 114 to store, for example,PGD 430 or to cache recently accessed summary pages for faster access.However, hash tables generally are set to a fixed size at the compiletime of the software/firmware operating on flash controller 104.Embodiments of the present invention provide dynamic sizing of hashtables, for example a hash table used to track the contents of the datacache, during operation of flash memory controller 104.

FIG. 12 shows a flow diagram of hash table size update operation 1200.At step 1202, buffer layer 210 (FIG. 2), which manages buffers 112 and114, initiates hash table size update operation 1200. For example,buffer layer 210 might run hash table size update operation 1200 atstartup of flash memory controller 104 or if the number of items trackedin the cache hash table changes during operation of flash memorycontroller 104. In embodiments of the present invention, the cache hashtable tracks a number of data chunks allocated to the cache, and hashtable size operation 1200 is performed when there is a change in thenumber of chunks tracked in the cache hash table. A change in the numberof chunks allocated to the cache, and thus tracked in the cache hashtable, might occur when flash media 118 is formatted.

Hash table size update operation 1200 might beneficially be performedduring operation of flash memory controller 104 since the size ofexternal RAM buffer 114 might not be a known, fixed value at the compiletime of software running on flash memory controller 104. For example,the size of RAM buffer 114, and thus the amount of RAM available tostore the cache and the cache hash table, might differ depending on theneeds of end users of flash memory controller 104. Additionally, asdescribed herein, embodiments of the present invention might employdifferent sector sizes of flash media 118 depending on the desired useof flash memory controller 104. Thus, the sector size of flash media 118might not be a known, fixed value at the compile time of softwarerunning on flash memory controller 104, since the sector size might bechanged if flash media 118 is re-formatted. Changes in sector sizeformatting of flash media 118 correspond to changes in the number ofdata chunks allocated to the cache and tracked by the cache hash table.

At step 1208, buffer layer 210 determines the number of items beingmanaged in the cache hash table, for example, by scanning the cache or,alternatively, FTL 208 might communicate to buffer layer 210 a desirednumber of items to be cached (e.g., the number of chunks allocated tothe cache). As described herein, the cache hash table might track anumber of data chunks allocated to the cache, and the LBA ranges of datachunks stored in the cache. At step 1212, if the number of actual ordesired number of cache items has reached a threshold, at step 1214buffer layer 210 sets the size of the cache hash table to acorresponding higher value, as will be described with regard to FIG. 13,and the process continues to step 1220. If, at step 1212, the number ofactual or desired number of cache hash table items has not reached amaximum threshold, at step 1218, buffer layer 210 sets the size of thecache hash table to a corresponding lower value, as will be describedwith regard to FIG. 13, and the process continues to step 1220. Once thesize of the cache hash table is set, for example by one of steps 1214(higher value) or 1218 (lower value), cache hash table size updateoperation 1200 is complete at step 1220. Thus, in comparison to afixed-size cache hash table at software/firmware compile time, the cachehash table resize threshold might be a fixed value at compile time andthe cache hash table itself might be resized as needed during theoperation of flash memory controller 104.

FIG. 13 shows an exemplary chart, 1300, of i) the actual size and ii)the number of items stored in an exemplary cache hash table asdynamically managed by cache hash table size update operation 1200. Asdescribed herein, an efficient hash table desirably is approximatelydouble the size of the number of items stored in the table, althoughother sizes might be employed (e.g., triple or quadruple the number ofitems). Further efficiency might be gained for indexing the hash tableif the total size is a power of 2. Embodiments of the present inventionmight set the cache hash table size (e.g., at steps 1214 and 1218 ofFIG. 12) equal to two times the number of items stored in the hashtable, rounded up or down to the nearest power of 2. For example, if 25chunks are allocated to the cache, the cache hash table will storetracking data corresponding to each of the 25 chunks. A hash table sizemight beneficially be set equal to double the number of chunks, in thiscase, 50. Embodiments of the present invention might instead set thehash table size equal to the nearest power of 2 to 50, which is 64 (2⁶),thus, the size of the hash table might be rounded up from 50 to 64.

For embodiments of the present invention, to perform this calculation,buffer layer 210 might double the most significant bit (MSB) of thenumber of items stored in the cache hash table (25₁₀=11001₂). Doublingjust the MSB of the number of items stored (10000₂=16₁₀;16*2=32₁₀=100000₂). However, as will be determined at the threshold testof step 1212, 64₁₀, not 32₁₀, is the nearest power of 2 to 50₁₀, whichis double the number chunks in the cache. The test of step 1212 areperformed by checking the second most significant bit (MSB) of thenumber of items stored in the cache hash table. When the second MSB isone, the test at step 1212 is true, and the cache hash table size is setto a corresponding size (e.g., rounded to the next higher power of 2) atstep 1214. Alternatively, when the second MSB is zero, the test at step1212 is false and the cache hash table size is decreased (e.g., roundedto the next lower power of 2) at step 1218. In the above example, thesecond MSB is one (25₁₀=11001₂) thus, at step 1218 the cache hash tablesize is set to the next higher power of 2, which is 64₁₀=1000000₂.

Further, the operations employed for the computations are relativelysimple; for example, a logical AND operation is performed on the numberof items stored and a bit mask, and the resulting number is left shiftedby one bit, resulting in twice the MSB. If the second MSB is one, theresulting number is left shifted again to equal the higher power of 2;otherwise, the resulting number is the nearest power of 2 and is used asthe hash table size. Although embodiments of the present invention testthe MSB and the second MSB of the number of items stored in the cachehash table to determine whether the threshold has been reached, othertests are possible. For example, the size of the cache hash table mightbe updated when a threshold number of entries is crossed between a lowerpower of 2 and a higher power of 2. For example, a hash table having asize N, where N is a power of 2, might be doubled in size when thenumber of chunks allocated to the cache exceeds N/2, or any other fixedvalue.

As described with regard to FIGS. 5 and 6, embodiments of the presentinvention might store portions of mapping data in a cache in RAM, forexample, to provide efficient performance of flash memory controller 104in i) sequential, ii) sequential streaming, and iii) limited rangerandom data transfers while operating with a relatively limited amountof RAM. Embodiments of the present invention might store in a RAM cache(e.g. at least one of buffers 112 and 114) one or more recently accessedsummary pages per each wear-level unit. The summary page cache mightemploy dynamic hash table sizing, as described with regard to FIGS. 12and 13. Described embodiments of the present invention might maintainthe summary page cache in order of most recently accessed summary pageto least recently accessed summary page, although other alternativestructures are possible.

FIG. 14 shows exemplary cache data structure 1400. FTL 208 might cacheone or more summary pages per each wear-level unit. As shown, each ofwear-level units 0-X have a corresponding cache, 1402(0)-1402(X), ofsummary pages, shown as summary pages 0-W. Thus, each cache1402(0)-1402(X) might store up to W summary pages associated with thewear-level unit. As will be described subsequently, each cache1402(0)-1402(X) might maintain a most-recently used list of summarypages for the wear-level unit corresponding to the cache. Cache entriesmight be “aged” such that the cache entries for the least recently usedsummary pages are “recycled” to add new summary pages to the cache.Embodiments of the present invention might “age” cache entries by savinga count of how often each entry is accessed. As described with regard toFIGS. 5 a and 6 a, when there is a cache hit (i.e. the summary page forthe requested LBA is stored in the cache), the summary page for therequested LBA is read from RAM and the requested data is read fromflash. When there is a cache miss (i.e. the summary page for therequested LBA is not stored in the cache), the summary page for therequested LBA is read from flash and the requested data is then readfrom flash.

FIG. 15 shows an exemplary cache for one wear level unit such as, forexample, cache 1402(0) of FIG. 14. As shown, cache 1402(0) might beimplemented as a linked list, starting at a head end, 1502, withsubsequent cache entries 1504, 1506, and so on, until final cache entry1510. Cache 1402(0) ends at a tail end, 1512. Thus, entry 1502 is the“first” entry in cache 1402(0), and entry 1510 is the “last” entry incache 1402(0). As will be described subsequently, a summary page cachednearer to head end 1502 has been accessed more recently, and a summarypage cached nearer to tail end 1512 has been accessed less recently. Asdescribed with regard to FIG. 14, cache 1402(0) might have W entries,where W is a positive integer. In exemplary embodiments of the presentinvention, W is equal to 3. The summary page caches, such as cache1402(0), are initialized during startup of flash memory storage system100. At startup, each cache entry 1504-1510 might be empty. A cacheentry might have one of three states: valid, pending, or empty.

As shown in FIG. 16, allocations of summary pages to cache 1402(0) ofFIG. 14 by FTL 208 begin by selecting the last cache entry. As shown,the last cache entry is allocated and its status is set to pending,shown as pending cache entry 1616. Although the exemplary case shown inFIG. 16 shows that the cache entries are empty, the allocation of newcache entries is substantially the same when the cache entries are full:the last cache entry is allocated and obtains pending status. A validpage near tail end 1512 is a less recently accessed entry and can bereplaced with a new cache entry. As shown in FIG. 16, head end 1502 isunlinked from cache entry 1604 and is linked to pending cache entry1616. Pending cache entry 1616 has been unlinked from tail end 1512, andtail end 1512 has been linked to the next closest cache entry, 1610.Thus, pending cache entry 1616 moves to the head end of cache 1402(0)since it is the most recently accessed summary page in cache 1402(0),and the next cache entry from the tail end, 1610, moves to the tail endof cache 1402(0) since it is the least recently accessed summary page incache 1402(0). A cache entry will maintain pending status until eitheri) the summary page is read from flash and loaded into the cache, or ii)an abort condition occurs. When the summary page is read from flash andloaded into the cache, the cache entry's status is updated from pendingto valid, as described below with respect to FIG. 17. When an abortcondition occurs, the cache entry's status is updated from pending toempty, as described below with respect to FIG. 19. In the event that allcache entries have pending status, any subsequent cache allocationrequests are denied until one or more of the pending cache entries havebeen processed.

FIG. 17 shows an exemplary case where a cache entry is updated frompending status to valid status when the summary page is read from flashand loaded into the cache. FIG. 17 shows a continuation of the exemplarycase of FIG. 16. As shown in FIG. 16, cache entry 1616 had pendingstatus and was at the head end of cache 1402(0). FIG. 17 shows thatbuffer layer 210 updates pending cache entry 1616 to valid status. Cacheentry 1616 remains at the head end of cache 1402(0). FIG. 18 showsanother exemplary case where a cache entry is updated from pendingstatus to valid status. As shown in FIG. 18, cache 1402(0) containsmultiple entries, shown as valid entries 1806 and 1808, and pendingentry 1804, as well as containing an empty entry, 1810. FTL 208 updatespending cache entry 1816 to valid status, and cache entry 1816 is movedto the head end of cache 1402(0) by unlinking cache entry 1816 fromcache entries 1804 and 1806. Cache entry 1804 is unlinked from head end1502 and is moved to the next position in the cache when cache entry1816 is linked to head end 1502 and cache entry 1804. Thus, as shown inFIGS. 17 and 18, whenever a cache entry is updated to valid status, thatcache entry is moved to the head end of the cache.

FIG. 19 shows an exemplary case where a pending cache entry is aborted.As shown in FIG. 19, FTL 208 aborts pending cache entry 1916. Theaborted cache entry is purged and returns to empty status. Empty entry1916 is moved to the end of cache 1402(0) by being linked to the tailend, 1512, of cache 1402(0). The empty entry is placed at the tail endof the cache and might be reused when a new summary page request isprocessed by FTL 208.

As described herein, a summary page might be updated as a result of awrite operation to flash media 118. When the summary page is updated,the new summary page is given an entry at the head end of cache 1402(0),shown as pending cache entry 2004. The previously cached version of thesummary page, shown as cache entry 2016, is stale, and FTL 208invalidates the entry, which returns to empty status and is moved to thetail end of cache 1402(0).

As described herein, flash memory controller 104 might temporarily storedata in RAM buffers. For example, some mapping data might be cached inRAM (e.g. at least one of buffers 112 and 114), for example, summarypages (e.g. summary page 420) might be cached as described with regardto FIG. 14. Further, data being read from, or written to, flash media118 might be cached in a buffer in RAM (e.g. at least one of buffers 112and 114), as described with regard to FIGS. 5 a and 6 a, respectively.Since some data might be cached in RAM, it is possible that data storedin the cache is “dirty”, meaning that data stored in the cache is morerecent than corresponding data stored in flash media 118. As describedherein, data transfers might be segmented into smaller internal datatransfers (“chunks”), where each chunk corresponds to a predefined LBArange (“sectors”).

FIG. 21 shows a state diagram, 2100, of the possible states of a cachedsector. Generally, buffer layer 210 manages the cache and status ofcached sectors. As shown, there are four possible states that a cachedsector might have: empty (state 2102), locked (state 2104), dirty (state2106), and valid (state 2108). A cached sector that does not contain anydata has empty state 2102, and maintains this state, as shown by statetransition 1, until the cached sector is requested as part of a hostoperation (e.g., for a read operation such as shown in FIG. 5 a or awrite operation such as shown in FIG. 6 a). Once a cached sector isrequested as part of a host operation, the sector obtains locked state2104 as indicated by state transition 2. In general, locked state 2104might indicate that the sector is currently being filled with data,either from the host or from the media. For example, if the hostoperation is a read operation (e.g. read operation 500 of FIG. 5 a),once the requested sector data is read from flash, the cached sectorobtains valid state 2108 as indicated by state transition 3. The cachedsector is valid because it contains the same data as stored in flashmedia 118. A valid cached sector might re-obtain locked state 2104, asindicated by state transition 4, if a subsequent write operation isrequested that includes the same sector. That sector would then obtaindirty state 2106, as indicated by state transition 5. The cached sectoris dirty because it contains more recent data than the sector stored inflash media 118.

If the host operation is a write operation (e.g., write operation 600 ofFIG. 6 a), once the requested sector data is provided from host layer202 to the cached sector, the cached sector obtains dirty status 2106,as indicated by state transition 5. The cached sector is dirty becauseit contains more recent data than the sector stored in flash media 118.A dirty cached sector obtains valid state 2108 when sector stored inflash media 118 is synchronized with the cached sector, as indicated bystate transition 6. The cache-media synchronization operation will bedescribed in greater detail with regard to FIG. 22. A dirty sector mightre-obtain locked status 2104, as indicated by state transition 4, if asubsequent write operation is requested that includes the same sector.That sector would then re-obtain dirty state 2106, as indicated by statetransition 5. As indicated by state transitions 5 and 8, a sector mighttransition between locked state 2104 and dirty state 2106 multiple timesbefore a cache-media synchronization occurs and the dirty cached sectorobtains valid state 2108 as indicated by state transition 6. A cachedsector having valid state 2108 or a cached sector having dirty state2106 might obtain empty status 2102, as indicated by state transitions 7and 9, respectively. A dirty or valid sector might become empty if, forexample, the sector is included in a range of data invalidated by bufferlayer 210. A range of data might be invalidated by buffer layer 210, forexample, when the read or write operation is complete and the buffer isdeallocated (e.g. step 518 of FIG. 5 a or step 618 of FIG. 6 a).

FIG. 22 shows cache-media synchronization operation 2200 performedgenerally by flash controller 104. At step 2202, cache-mediasynchronization is initiated, for example, during idle time of flashmemory storage system 100. At step 2204, buffer layer 210 scans a chunktable, as will be described with regard to FIG. 23, to find “dirty”chunks. Dirty chunks are chunks having at least one sector stored in thecache that is more recent than the corresponding sector stored in flashmedia 118. As shown in FIG. 23, dirty chunks are generally tracked byupdating the data stored in a dirty sector bitmask corresponding to eachchunk.

FIG. 23 shows a table of dirty sector bitmasks 2306(1)-2306(Z) employedin cache-media synchronization. As shown, dirty sector bitmasks2306(1)-2306(Z) correspond to chunks 2302(1)-2302(Z), where Z is thetotal number of chunks stored in the cache. As described herein, a chunkmight correspond to a fixed LBA range of contiguous sectors. Dirtysector bitmasks 2306(1)-2306(Z) might generally correspond to the stateof a sector, as described with regard to FIG. 21. Dirty sector bitmasks2306(1)-2306(Z) track the status of individual sectors within each chunk2302(1)-2302(Z). For example, each dirty sector bitmask 2306(1)-2306(Z)has a bit corresponding to each sector in each respective chunk. A “0”in the bit mask might indicate that the cached sector and the sectorstored in flash are synchronized; while a “1” in the bit mask mightindicate that the sector is dirty (i.e. the cached sector is more recentthan the sector stored in flash). As shown in exemplary bitmasks of FIG.23, each chunk corresponds to 8 sectors (i.e. dirty sector bitmasks2306(1)-2306(Z) are 8 bits long).

Referring back to FIG. 22, at step 2206, buffer layer 210 scans dirtybitmasks 2306(1)-2306(Z) to determine if any chunks stored in the cacheare “dirty”. For example, embodiments of the present invention mightcheck if each of bitmasks 2306(1)-2306(Z) is nonzero to determine if achunk contains dirty sectors. If the current chunk is not dirty, at step2208, buffer layer 210 scans the next chunk and the process returns tothe test of step 2206. If the current chunk is dirty, at step 2210,buffer layer 210 adds the chunk to a list of dirty chunks to be sent toFTL 208 to be written to flash media 118. At step 2212, if there arecached chunks remaining to be scanned, at step 2208, buffer layer 210scans the next chunk and the process returns back to the test of step2206. If the last chunk, Z, stored in the cache has been scanned, theprocess advances to step 2214 where buffer layer 210 provides the dataof the dirty chunks to FTL 208 to be written to flash media 118 (e.g.the write operation of FIG. 6 a). FTL 208 might optionally confirm thatthe data was written by reading back the written sectors. At step 2216,buffer layer 210 clears the dirty sector bitmasks 2306(1)-2306(Z) ofFIG. 23, and cache-media synchronization is complete at step 2218.

Thus, in embodiments of the present invention, buffer layer 210 mightsynchronize dirty sectors at a coarse, or “fuzzy”, level rather thansynchronizing each individual sector. For example, buffer layer 210merely checks whether dirty sector mask bitmasks 2306(1)-2306(Z) arenonzero, which includes all cases where one or more sectors are dirty.However, buffer layer 210 does not track each individual sector or trackhow many sectors within a chunk must be updated. Thus, buffer layer 210might reduce its overhead in controlling cache-media synchronization byonly performing synchronization of dirty cache data at a chunk levelrather than at a sector level. Buffer layer 210 might send entire chunksof data (possibly including some combination of dirty, valid and emptysectors) to FTL 208 to be written to flash media 118.

Reference herein to “one embodiment” or “an embodiment” means that aparticular feature, structure, or characteristic described in connectionwith the embodiment can be included in at least one embodiment of theinvention. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment, nor are separate or alternative embodiments necessarilymutually exclusive of other embodiments. The same applies to the term“implementation.”

While the exemplary embodiments of the present invention have beendescribed with respect to processing blocks in a software program,including possible implementation as a digital signal processor,micro-controller, or general purpose computer, the present invention isnot so limited. As would be apparent to one skilled in the art, variousfunctions of software may also be implemented as processes of circuits.Such circuits may be employed in, for example, a single integratedcircuit, a multi-chip module, a single card, or a multi-card circuitpack.

The present invention can be embodied in the form of methods andapparatuses for practicing those methods. The present invention can alsobe embodied in the form of program code embodied in tangible media, suchas magnetic recording media, optical recording media, solid statememory, floppy diskettes, CD-ROMs, hard drives, or any othernon-transitory machine-readable storage medium, wherein, when theprogram code is loaded into and executed by a machine, such as acomputer, the machine becomes an apparatus for practicing the invention.The present invention can also be embodied in the form of program code,for example, whether stored in a non-transitory machine-readable storagemedium, loaded into and/or executed by a machine, or transmitted oversome transmission medium or carrier, such as over electrical wiring orcabling, through fiber optics, or via electromagnetic radiation,wherein, when the program code is loaded into and executed by a machine,such as a computer, the machine becomes an apparatus for practicing theinvention. When implemented on a general-purpose processor, the programcode segments combine with the processor to provide a unique device thatoperates analogously to specific logic circuits. The present inventioncan also be embodied in the form of a bitstream or other sequence ofsignal values electrically or optically transmitted through a medium,stored magnetic-field variations in a magnetic recording medium, etc.,generated using a method and/or an apparatus of the present invention.

It should be understood that the steps of the exemplary methods setforth herein are not necessarily required to be performed in the orderdescribed, and the order of the steps of such methods should beunderstood to be merely exemplary. Likewise, additional steps may beincluded in such methods, and certain steps may be omitted or combined,in methods consistent with various embodiments of the present invention.

As used herein in reference to an element and a standard, the term“compatible” means that the element communicates with other elements ina manner wholly or partially specified by the standard, and would berecognized by other elements as sufficiently capable of communicatingwith the other elements in the manner specified by the standard. Thecompatible element does not need to operate internally in a mannerspecified by the standard.

Also for purposes of this description, the terms “couple,” “coupling,”“coupled,” “connect,” “connecting,” or “connected” refer to any mannerknown in the art or later developed in which energy is allowed to betransferred between two or more elements, and the interposition of oneor more additional elements is contemplated, although not required.Conversely, the terms “directly coupled,” “directly connected,” etc.,imply the absence of such additional elements. Signals and correspondingnodes or ports may be referred to by the same name and areinterchangeable for purposes here.

It will be further understood that various changes in the details,materials, and arrangements of the parts which have been described andillustrated in order to explain the nature of this invention may be madeby those skilled in the art without departing from the scope of theinvention as expressed in the following claims.

1. A method of transferring, by a media controller, data associated witha host data transfer between a host device and a storage media, themethod comprising: by a buffer layer module of the media controller:segmenting the host data transfer into one or more data transfersegments, wherein each data transfer segment corresponds to at least aportion of the data; allocating a number of physical buffers to avirtual circular buffer for buffering the one or more data transfersegments; and transferring, by the virtual circular buffer, each of thedata transfer segments between the host device and the storage mediathrough the allocated physical buffers.
 2. The invention of claim 1,wherein the step of transferring, by the virtual circular buffer, eachof the data transfer segments comprises: if the host data transfer is ahost write operation: a) providing, by a host layer module of the mediacontroller, an initial number of data transfer segments to correspondingones of the allocated physical buffers, the number of data transfersegments equivalent to the number of allocated physical buffers; b)providing, by the buffer layer module, the number of data transfersegments to the storage media; and c) repeating steps a) and b) for eachsubsequent number of data transfer segments that remain; otherwise, ifthe host data transfer is a host read operation: d) transferring, by thebuffer layer module, an initial number of data transfer segments fromthe storage media to corresponding ones of the allocated physicalbuffers, the number of data transfer segments equivalent to the numberof allocated physical buffers; e) providing, by the buffer layer module,the number of data transfer segments to the host layer module fortransmission to the host device; and f) repeating steps d) and e) foreach subsequent number of data transfer segments that remain.
 3. Theinvention of claim 2, further comprising: if the host data transfer is ahost write operation: performing, in parallel, i) a media-side transferof step b) for a first number of data transfer segments, and ii) ahost-side transfer of step a) for a second number of data transfersegments; otherwise, if the host data transfer is a host read operation:performing, in parallel, i) a host-side transfer of step e) for a firstnumber of data transfer segments, and ii) a media-side transfer of stepd) for a second number of data transfer segments.
 4. The invention ofclaim 2, comprising reusing the allocated physical buffers of thevirtual circular buffer to buffer each subsequent number of datatransfer segments.
 5. The invention of claim 2, comprising allocatingalternate physical buffers to the virtual circular buffer to buffer eachsubsequent number of data transfer segments.
 6. The invention of claim1, wherein, for the method, the storage media comprises flash memory,the flash memory comprising one or more flash memory dies, and whereinone or more virtual circular buffers each correspond to one of the oneor more flash memory dies.
 7. The invention of claim 6, furthercomprising: performing, in parallel, one or more host data transfers bythe one or more virtual circular buffers, each of the host datatransfers between the host device and the one or more flash memory diestransferred through the allocated physical buffers of the one or morevirtual circular buffers.
 8. The invention of claim 1, wherein theinitial number of allocated physical buffers is two.
 9. The invention ofclaim 8, further comprising: if the media controller has availablebuffer resources, allocating one or more additional physical buffers tobuffer the data transfer segments.
 10. The invention of claim 1, whereinthe size of the data transfer segments is 256 kB.
 11. A machine-readablestorage medium, having encoded thereon program code, wherein, when theprogram code is executed by a machine, the machine implements a methodof transferring, by a media controller, data associated with a host datatransfer between a host device and a storage media, the methodcomprising: by a buffer layer module of the media controller: segmentingthe host data transfer into one or more data transfer segments, whereineach data transfer segment corresponds to at least a portion of thedata; allocating a number of physical buffers to a virtual circularbuffer for buffering the one or more data transfer segments; andtransferring, by the virtual circular buffer, each of the data transfersegments between the host device and the storage media through theallocated physical buffers.
 12. The invention of claim 11, wherein thestep of transferring, by the virtual circular buffer, each of the datatransfer segments comprises: if the host data transfer is a host writeoperation: a) providing, by a host layer module of the media controller,an initial number of data transfer segments to corresponding ones of theallocated physical buffers, the number of data transfer segmentsequivalent to the number of allocated physical buffers; b) providing, bythe buffer layer module, the number of data transfer segments to thestorage media; and c) repeating steps a) and b) for each subsequentnumber of data transfer segments that remain; otherwise, if the hostdata transfer is a host read operation: d) transferring, by the bufferlayer module, an initial number of data transfer segments from thestorage media to corresponding ones of the allocated physical buffers,the number of data transfer segments equivalent to the number ofallocated physical buffers; e) providing, by the buffer layer module,the number of data transfer segments to the host layer module fortransmission to the host device; and f) repeating steps d) and e) foreach subsequent number of data transfer segments that remain.
 13. Theinvention of claim 12, further comprising: if the host data transfer isa host write operation: performing, in parallel, i) a media-sidetransfer of step b) for a first number of data transfer segments, andii) a host-side transfer of step a) for a second number of data transfersegments; otherwise, if the host data transfer is a host read operation:performing, in parallel, i) a host-side transfer of step e) for a firstnumber of data transfer segments, and ii) a media-side transfer of stepd) for a second number of data transfer segments.
 14. The invention ofclaim 12, comprising reusing the allocated physical buffers of thevirtual circular buffer to buffer each subsequent number of datatransfer segments.
 15. The invention of claim 12, comprising allocatingalternate physical buffers to the virtual circular buffer to buffer eachsubsequent number of data transfer segments.
 16. The invention of claim11, wherein, for the method, the storage media comprises flash memory,the flash memory comprising one or more flash memory dies, and whereinone or more virtual circular buffers each correspond to one of the oneor more flash memory dies.
 17. The invention of claim 16, furthercomprising: performing, in parallel, one or more host data transfers bythe one or more virtual circular buffers, each of the host datatransfers between the host device and the one or more flash memory diestransferred through the allocated physical buffers of the one or morevirtual circular buffers.
 18. The invention of claim 11, furthercomprising: if the media controller has available buffer resources,allocating one or more additional physical buffers to buffer the datatransfer segments.
 19. A media controller that transfers data associatedwith a host data transfer between a host device and at least one storagemedia, the media controller comprising: a host layer module adapted toprocess the host data transfer; a buffer layer module adapted to i)segment the host data transfer into one or more data transfer segments,wherein each data transfer segment corresponds to at least a portion ofthe data, and ii) allocate one or more physical buffers to a virtualcircular buffer; wherein, if the host data transfer is a host writeoperation: the host layer module transfers an initial number of datatransfer segments to corresponding ones of an initial number ofallocated physical buffers of the virtual circular buffer, the initialnumber of data transfer segments equivalent to the initial number ofallocated physical buffers; the buffer layer module transfers, inparallel, i) the initial number of data transfer segments to the storagemedia from the virtual circular buffer, and ii) one or more subsequentnumbers of data transfer segments from the host layer module tocorresponding ones of allocated physical buffers of the virtual circularbuffer for remaining data transfer segments; otherwise, if the host datatransfer is a host read operation: the buffer layer module transfers aninitial number of data transfer segments from the storage media tocorresponding ones of the allocated physical buffers of the virtualcircular buffer, the initial number of data transfer segments equivalentto the initial number of allocated physical buffers; and the bufferlayer module transfers, in parallel, i) the initial number of datatransfer segments from the virtual circular buffer to the host layermodule, and ii) one or more subsequent numbers of data transfer segmentsfrom storage media to subsequent corresponding ones of allocatedphysical buffers of the virtual circular buffer for remaining datatransfer segments.
 20. The invention of claim 19, wherein the apparatusis implemented in a monolithic integrated circuit chip.